Data-Driven Application Maintenance: Views from the Trenches

In this paper we present our experience during design, development, and pilot deployments of a data-driven machine learning based application maintenance solution. We implemented a proof of concept to address a spectrum of interrelated problems encountered in application maintenance projects including duplicate incident ticket identification, assignee recommendation, theme mining, and mapping of incidents to business processes. In the context of IT services, these problems are frequently encountered, yet there is a gap in bringing automation and optimization. Despite long-standing research around mining and analysis of software repositories, such research outputs are not adopted well in practice due to the constraints these solutions impose on the users. We discuss need for designing pragmatic solutions with low barriers to adoption and addressing right level of complexity of problems with respect to underlying business constraints and nature of data.Comment: Earlier version of paper appearing in proceedings of the 4th International Workshop on Software Engineering Research and Industrial Practice (SER&IP), IEEE Press, pp. 48-54, 201

Identifying Implementation Bugs in Machine Learning based Image Classifiers using Metamorphic Testing

We have recently witnessed tremendous success of Machine Learning (ML) in practical applications. Computer vision, speech recognition and language translation have all seen a near human level performance. We expect, in the near future, most business applications will have some form of ML. However, testing such applications is extremely challenging and would be very expensive if we follow today's methodologies. In this work, we present an articulation of the challenges in testing ML based applications. We then present our solution approach, based on the concept of Metamorphic Testing, which aims to identify implementation bugs in ML based image classifiers. We have developed metamorphic relations for an application based on Support Vector Machine and a Deep Learning based application. Empirical validation showed that our approach was able to catch 71% of the implementation bugs in the ML applications.Comment: Published at 27th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2018

Global, regional, and national burden of stroke and its risk factors, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019

Background Regularly updated data on stroke and its pathological types, including data on their incidence, prevalence, mortality, disability, risk factors, and epidemiological trends, are important for evidence-based stroke care planning and resource allocation. The Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) aims to provide a standardised and comprehensive measurement of these metrics at global, regional, and national levels. Methods We applied GBD 2019 analytical tools to calculate stroke incidence, prevalence, mortality, disability-adjusted life-years (DALYs), and the population attributable fraction (PAF) of DALYs (with corresponding 95% uncertainty intervals [UIs]) associated with 19 risk factors, for 204 countries and territories from 1990 to 2019. These estimates were provided for ischaemic stroke, intracerebral haemorrhage, subarachnoid haemorrhage, and all strokes combined, and stratified by sex, age group, and World Bank country income level. Findings In 2019, there were 12·2 million (95% UI 11·0–13·6) incident cases of stroke, 101 million (93·2–111) prevalent cases of stroke, 143 million (133–153) DALYs due to stroke, and 6·55 million (6·00–7·02) deaths from stroke. Globally, stroke remained the second-leading cause of death (11·6% [10·8–12·2] of total deaths) and the third-leading cause of death and disability combined (5·7% [5·1–6·2] of total DALYs) in 2019. From 1990 to 2019, the absolute number of incident strokes increased by 70·0% (67·0–73·0), prevalent strokes increased by 85·0% (83·0–88·0), deaths from stroke increased by 43·0% (31·0–55·0), and DALYs due to stroke increased by 32·0% (22·0–42·0). During the same period, age-standardised rates of stroke incidence decreased by 17·0% (15·0–18·0), mortality decreased by 36·0% (31·0–42·0), prevalence decreased by 6·0% (5·0–7·0), and DALYs decreased by 36·0% (31·0–42·0). However, among people younger than 70 years, prevalence rates increased by 22·0% (21·0–24·0) and incidence rates increased by 15·0% (12·0–18·0). In 2019, the age-standardised stroke-related mortality rate was 3·6 (3·5–3·8) times higher in the World Bank low-income group than in the World Bank high-income group, and the age-standardised stroke-related DALY rate was 3·7 (3·5–3·9) times higher in the low-income group than the high-income group. Ischaemic stroke constituted 62·4% of all incident strokes in 2019 (7·63 million [6·57–8·96]), while intracerebral haemorrhage constituted 27·9% (3·41 million [2·97–3·91]) and subarachnoid haemorrhage constituted 9·7% (1·18 million [1·01–1·39]). In 2019, the five leading risk factors for stroke were high systolic blood pressure (contributing to 79·6 million [67·7–90·8] DALYs or 55·5% [48·2–62·0] of total stroke DALYs), high body-mass index (34·9 million [22·3–48·6] DALYs or 24·3% [15·7–33·2]), high fasting plasma glucose (28·9 million [19·8–41·5] DALYs or 20·2% [13·8–29·1]), ambient particulate matter pollution (28·7 million [23·4–33·4] DALYs or 20·1% [16·6–23·0]), and smoking (25·3 million [22·6–28·2] DALYs or 17·6% [16·4–19·0]). Interpretation The annual number of strokes and deaths due to stroke increased substantially from 1990 to 2019, despite substantial reductions in age-standardised rates, particularly among people older than 70 years. The highest age-standardised stroke-related mortality and DALY rates were in the World Bank low-income group. The fastest-growing risk factor for stroke between 1990 and 2019 was high body-mass index. Without urgent implementation of effective primary prevention strategies, the stroke burden will probably continue to grow across the world, particularly in low-income countries.publishedVersio

Global, regional, and national burden of stroke and its risk factors, 1990-2019: a systematic analysis for the Global Burden of Disease Study 2019

Background Regularly updated data on stroke and its pathological types, including data on their incidence, prevalence, mortality, disability, risk factors, and epidemiological trends, are important for evidence-based stroke care planning and resource allocation. The Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) aims to provide a standardised and comprehensive measurement of these metrics at global, regional, and national levels. Methods We applied GBD 2019 analytical tools to calculate stroke incidence, prevalence, mortality, disability-adjusted life-years (DALYs), and the population attributable fraction (PAF) of DALYs (with corresponding 95% uncertainty intervals UIs]) associated with 19 risk factors, for 204 countries and territories from 1990 to 2019. These estimates were provided for ischaemic stroke, intracerebral haemorrhage, subarachnoid haemorrhage, and all strokes combined, and stratified by sex, age group, and World Bank country income level. Findings In 2019, there were 12.2 million (95% UI 11.0-13.6) incident cases of stroke, 101 million (93.2-111) prevalent cases of stroke, 143 million (133-153) DALYs due to stroke, and 6.55 million (6.00-7.02) deaths from stroke. Globally, stroke remained the second-leading cause of death (11.6% 10.8-12.2] of total deaths) and the third-leading cause of death and disability combined (5.7% 5.1-6.2] of total DALYs) in 2019. From 1990 to 2019, the absolute number of incident strokes increased by 70.0% (67.0-73.0), prevalent strokes increased by 85.0% (83.0-88.0), deaths from stroke increased by 43.0% (31.0-55.0), and DALYs due to stroke increased by 32.0% (22.0-42.0). During the same period, age-standardised rates of stroke incidence decreased by 17.0% (15.0-18.0), mortality decreased by 36.0% (31.0-42.0), prevalence decreased by 6.0% (5.0-7.0), and DALYs decreased by 36.0% (31.0-42.0). However, among people younger than 70 years, prevalence rates increased by 22.0% (21.0-24.0) and incidence rates increased by 15.0% (12.0-18.0). In 2019, the age-standardised stroke-related mortality rate was 3.6 (3.5-3.8) times higher in the World Bank low-income group than in the World Bank high-income group, and the age-standardised stroke-related DALY rate was 3.7 (3.5-3.9) times higher in the low-income group than the high-income group. Ischaemic stroke constituted 62.4% of all incident strokes in 2019 (7.63 million 6.57-8.96]), while intracerebral haemorrhage constituted 27.9% (3.41 million 2.97-3.91]) and subarachnoid haemorrhage constituted 9.7% (1.18 million 1.01-1.39]). In 2019, the five leading risk factors for stroke were high systolic blood pressure (contributing to 79.6 million 67.7-90.8] DALYs or 55.5% 48.2-62.0] of total stroke DALYs), high body-mass index (34.9 million 22.3-48.6] DALYs or 24.3% 15.7-33.2]), high fasting plasma glucose (28.9 million 19.8-41.5] DALYs or 20.2% 13.8-29.1]), ambient particulate matter pollution (28.7 million 23.4-33.4] DALYs or 20.1% 16.6-23.0]), and smoking (25.3 million 22.6-28.2] DALYs or 17.6% 16.4-19.0]). Interpretation The annual number of strokes and deaths due to stroke increased substantially from 1990 to 2019, despite substantial reductions in age-standardised rates, particularly among people older than 70 years. The highest age-standardised stroke-related mortality and DALY rates were in the World Bank low-income group. The fastest-growing risk factor for stroke between 1990 and 2019 was high body-mass index. Without urgent implementation of effective primary prevention strategies, the stroke burden will probably continue to grow across the world, particularly in low-income countries

Cancer Incidence, Mortality, Years of Life Lost, Years Lived With Disability, and Disability-Adjusted Life Years for 29 Cancer Groups From 2010 to 2019: A Systematic Analysis for the Global Burden of Disease Study 2019.

The Global Burden of Diseases, Injuries, and Risk Factors Study 2019 (GBD 2019) provided systematic estimates of incidence, morbidity, and mortality to inform local and international efforts toward reducing cancer burden. To estimate cancer burden and trends globally for 204 countries and territories and by Sociodemographic Index (SDI) quintiles from 2010 to 2019. The GBD 2019 estimation methods were used to describe cancer incidence, mortality, years lived with disability, years of life lost, and disability-adjusted life years (DALYs) in 2019 and over the past decade. Estimates are also provided by quintiles of the SDI, a composite measure of educational attainment, income per capita, and total fertility rate for those younger than 25 years. Estimates include 95% uncertainty intervals (UIs). In 2019, there were an estimated 23.6 million (95% UI, 22.2-24.9 million) new cancer cases (17.2 million when excluding nonmelanoma skin cancer) and 10.0 million (95% UI, 9.36-10.6 million) cancer deaths globally, with an estimated 250 million (235-264 million) DALYs due to cancer. Since 2010, these represented a 26.3% (95% UI, 20.3%-32.3%) increase in new cases, a 20.9% (95% UI, 14.2%-27.6%) increase in deaths, and a 16.0% (95% UI, 9.3%-22.8%) increase in DALYs. Among 22 groups of diseases and injuries in the GBD 2019 study, cancer was second only to cardiovascular diseases for the number of deaths, years of life lost, and DALYs globally in 2019. Cancer burden differed across SDI quintiles. The proportion of years lived with disability that contributed to DALYs increased with SDI, ranging from 1.4% (1.1%-1.8%) in the low SDI quintile to 5.7% (4.2%-7.1%) in the high SDI quintile. While the high SDI quintile had the highest number of new cases in 2019, the middle SDI quintile had the highest number of cancer deaths and DALYs. From 2010 to 2019, the largest percentage increase in the numbers of cases and deaths occurred in the low and low-middle SDI quintiles. The results of this systematic analysis suggest that the global burden of cancer is substantial and growing, with burden differing by SDI. These results provide comprehensive and comparable estimates that can potentially inform efforts toward equitable cancer control around the world.Funding/Support: The Institute for Health Metrics and Evaluation received funding from the Bill & Melinda Gates Foundation and the American Lebanese Syrian Associated Charities. Dr Aljunid acknowledges the Department of Health Policy and Management of Kuwait University and the International Centre for Casemix and Clinical Coding, National University of Malaysia for the approval and support to participate in this research project. Dr Bhaskar acknowledges institutional support from the NSW Ministry of Health and NSW Health Pathology. Dr Bärnighausen was supported by the Alexander von Humboldt Foundation through the Alexander von Humboldt Professor award, which is funded by the German Federal Ministry of Education and Research. Dr Braithwaite acknowledges funding from the National Institutes of Health/ National Cancer Institute. Dr Conde acknowledges financial support from the European Research Council ERC Starting Grant agreement No 848325. Dr Costa acknowledges her grant (SFRH/BHD/110001/2015), received by Portuguese national funds through Fundação para a Ciência e Tecnologia, IP under the Norma Transitória grant DL57/2016/CP1334/CT0006. Dr Ghith acknowledges support from a grant from Novo Nordisk Foundation (NNF16OC0021856). Dr Glasbey is supported by a National Institute of Health Research Doctoral Research Fellowship. Dr Vivek Kumar Gupta acknowledges funding support from National Health and Medical Research Council Australia. Dr Haque thanks Jazan University, Saudi Arabia for providing access to the Saudi Digital Library for this research study. Drs Herteliu, Pana, and Ausloos are partially supported by a grant of the Romanian National Authority for Scientific Research and Innovation, CNDS-UEFISCDI, project number PN-III-P4-ID-PCCF-2016-0084. Dr Hugo received support from the Higher Education Improvement Coordination of the Brazilian Ministry of Education for a sabbatical period at the Institute for Health Metrics and Evaluation, between September 2019 and August 2020. Dr Sheikh Mohammed Shariful Islam acknowledges funding by a National Heart Foundation of Australia Fellowship and National Health and Medical Research Council Emerging Leadership Fellowship. Dr Jakovljevic acknowledges support through grant OI 175014 of the Ministry of Education Science and Technological Development of the Republic of Serbia. Dr Katikireddi acknowledges funding from a NHS Research Scotland Senior Clinical Fellowship (SCAF/15/02), the Medical Research Council (MC_UU_00022/2), and the Scottish Government Chief Scientist Office (SPHSU17). Dr Md Nuruzzaman Khan acknowledges the support of Jatiya Kabi Kazi Nazrul Islam University, Bangladesh. Dr Yun Jin Kim was supported by the Research Management Centre, Xiamen University Malaysia (XMUMRF/2020-C6/ITCM/0004). Dr Koulmane Laxminarayana acknowledges institutional support from Manipal Academy of Higher Education. Dr Landires is a member of the Sistema Nacional de Investigación, which is supported by Panama’s Secretaría Nacional de Ciencia, Tecnología e Innovación. Dr Loureiro was supported by national funds through Fundação para a Ciência e Tecnologia under the Scientific Employment Stimulus–Institutional Call (CEECINST/00049/2018). Dr Molokhia is supported by the National Institute for Health Research Biomedical Research Center at Guy’s and St Thomas’ National Health Service Foundation Trust and King’s College London. Dr Moosavi appreciates NIGEB's support. Dr Pati acknowledges support from the SIAN Institute, Association for Biodiversity Conservation & Research. Dr Rakovac acknowledges a grant from the government of the Russian Federation in the context of World Health Organization Noncommunicable Diseases Office. Dr Samy was supported by a fellowship from the Egyptian Fulbright Mission Program. Dr Sheikh acknowledges support from Health Data Research UK. Drs Adithi Shetty and Unnikrishnan acknowledge support given by Kasturba Medical College, Mangalore, Manipal Academy of Higher Education. Dr Pavanchand H. Shetty acknowledges Manipal Academy of Higher Education for their research support. Dr Diego Augusto Santos Silva was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil Finance Code 001 and is supported in part by CNPq (302028/2018-8). Dr Zhu acknowledges the Cancer Prevention and Research Institute of Texas grant RP210042